Information technology environments can include various types of edge devices. In general, an edge device is an electronic device that can form an endpoint of a network connection. An edge device can be a device on an Internet-of-Things (IoT) (an “IoT device”), that can collect data and exchange data on a network. An IoT device can be connected to the network permanently or intermittently. In some cases, an IoT device may include electronics, software, sensors, and network connectivity components included in other devices, vehicles, buildings, or other items. An edge device may perform machine-to-machine (M2M) communications directly with other devices (e.g., device-to-device communication) over a network and may also physically interact with its environment.
Multiple edge devices within an information technology environment can generate large amounts of data (“edge data”) from diverse locations. The edge data may be generated passively (e.g., sensors collecting environmental temperatures) and/or generated actively (e.g., cameras photographing detected objects). The edge data may include machine-generated data (“machine data”), which can include performance data, diagnostic information, or any other data that can be analyzed to diagnose equipment performance problems, monitor user interactions, and to derive other insights. The large amounts and often-diverse nature of edge data in certain environments can give rise to various challenges in relation to managing, understanding and effectively utilizing the data.
A number of tools are available to analyze data generated by edge devices. To reduce the volume of the potentially vast amount of edge data that may be generated, edge data may be pre-processed based on anticipated data-analysis needs. For example, specified data items may be extracted from the edge data and stored in a database to facilitate efficient retrieval and analysis of those data items at a later time. The remainder of the generated edge data typically is not saved and is discarded during pre-processing. However, as storage capacity becomes progressively less expensive and more plentiful, storing massive quantities of minimally processed or unprocessed data (collectively and individually referred to as “raw data”) for later retrieval and analysis is becoming increasingly more feasible.
In general, storing raw edge data and performing analysis on that data later (i.e., at “search time”) can provide greater flexibility because it enables analysis of all of the generated edge data instead of only a small subset of it. This may, for example, enable an analyst to investigate different aspects of the edge data that previously were unavailable for analysis because massive amounts of edge data were discarded.
However, storing and analyzing massive quantities of edge data presents a number of challenges. For example, implementing edge analytics is a computationally intensive process that can push the limits of edge devices that have limited storage and computational capabilities. Moreover, the analytics tools implemented by edge devices fail to benefit from their interconnectedness with other devices.